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Abstract 

The current study explored the extent to which a common measure of perceived implementation 
of critical features of School-wide Positive Behavior Support (SWPBS) predicted fidelity of 
implementation 3 years later. Respondents included school personnel from 261 schools across 
the United States implementing SWPBS. School teams completed the PBIS Self-Assessment 
Surx’ey to self-assess fidelity of implementation in different SWPBS settings (school-wide, non- 
classroom, classroom, individual). These scores were then analyzed to assess whether certain 
items predicted the fidelity of SWPBS implementation, as assessed through another fidelity of 
implementation measure, the School-wide Benchmarks of Quality, 3 years later. Regression 
analyses indicated that self-reported fidelity of implementation of Classrooms Systems 
significantly predicted both sustained implementation and student outcomes, as assessed through 
levels of Office Discipline Referrals. Within Classroom Systems, regular acknowledgement of 
expected behaviors, matching instruction to student ability, and access to additional support were 
the strongest predictors of sustained implementation. Results are discussed in terms of critical 
areas for focusing SWPBS training to increase the likelihood of sustained implementation. 

Keywords: Positive Behavior Support, prevention, sustainability, implementation science 
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Critical Features Predicting Sustained Implementation of 
School-wide Positive Behavior Support 

Schools are complex host environments that serve a diverse student population with 
unique skills, strengths, and challenges (Kame’enui, Simmons, & Coyne, 2000). Because 
students spend more than 14,000 hours in the school environment from Kindergarten to Grade 
12, school personnel have prime opportunities to implement and sustain practices that promote 
academic, social-emotional, and behavioral success of their students. School-wide Positive 
Behavior Support (SWPBS; Sugai & Horner, 2006) is a preventive approach that has been 
adopted by over 14,000 schools in the last 15 years (Center on Positive Behavioral Interventions 
and Supports, 2012) and has been associated with valued outcomes such as (a) decreases in 
office discipline referrals, suspensions, and expulsions, (b) increases in classroom instructional 
time, positive student-teacher interactions, academic achievement, and social competency, and 
(c) improvement in social climate (e.g., Algozzine & Algozzine, 2007; Bradshaw, Mitchell, & 
Leaf, 2010; Luiselli, Putnam, Handler, & Feinberg, 2005; Nelson, 1996; Scott & Barrett, 2004). 
Typical implementation of SWPBS includes addressing the school-wide, non-classroom, 
classroom, and individual settings (Sugai, Horner, & Todd, 2000). All school personnel are 
involved in establishing the school-wide framework of behavior support, which provides a 
common foundation that is then incorporated into classroom and non-classroom settings (Lewis 
& Sugai, 1999). However, implementation is often dependent on individual classroom teachers, 
whose regular interactions with students are intended to be consistent with the critical features of 
the approach (Han & Weiss, 2005). 

The Importance of Sustaining Effective School-wide Interventions 

Han and Weiss (2005) defined sustainability as the continued implementation of a 
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practice with ongoing fidelity of implementation to the core program principles, after 
supplemental resources used to support initial training and implementation are withdrawn. The 
continual reimplementation of different interventions every few years is costly (e.g., loss of 
money, time) and breeds negative attitudes and resistance in school personnel in implementing 
new practices (Greenberg, Weissburg, & O'Brien, 2003). Thus, sustained implementation of 
effective, multiyear, school-based preventative practices should be identified as a key goal at the 
onset of planning activities (Pluye, Potvin, Denis, Pelletier, & Mannoni, 2005). 

Because fidelity of implementation (the extent to which the intervention is delivered as 
intended) is the mechanism by which valued outcomes are obtained, fidelity becomes critical in 
sustainability. Without adequate fidelity, the practice will no longer be capable of achieving 
those outcomes, and continued implementation will no longer be reinforced in this way (Andreou 
& McIntosh, 2013; McIntosh, Homer, & Sugai, 2009). Accordingly, identifying variables that 
influence fidelity of implementation represents an important area for sustainability research. 
When such variables are malleable, they represent important targets for researchers and 
practitioners alike in enhancing the sustainability of effective interventions. 

Variables Related to Fidelity and Sustainability 

A few recent studies have identified variables that appear to be important to sustained 
fidelity of SWPBS implementation, such as staff buy-in, administrative support, implementer 
skill, effective teaming, the use of data for decision-making, and ongoing technical assistance 
(Coffey & Homer, 2012; McIntosh, Mercer, et al., in press; McIntosh, Predy, et al., in press). 

Staff buy-in. Implementer acceptance and commitment to the practice is an essential 
feature contributing to SWPBS sustainability (Coffey & Horner, 2012). Practices that are time- 
efficient (Witt, Martens, & Elliott, 1984), address an identified gap in existing services (Han & 
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Weiss, 2005), and meet the diverse needs of all students in the classroom (Gersten, Chard, & 
Baker, 2000) are more likely to be seen as acceptable. However, implementers may refuse to 
implement the practice with fidelity if they deem it unacceptable to their values or those of the 
recipients, regardless of whether the practice is empirically proven to improve student outcomes 
(Witt et al., 1984). For example, some educators may believe that they are already meeting the 
behavior needs of their students without needing to change their practice (McKevitt & Braaksma, 
2008) or that positive reinforcement for appropriate behavior is unacceptable and possibly 
harmful (Lohrmann, Forman, Martin, & Palmieri, 2008). Sugai and Homer (2006) have 
suggested that SWPBS implementation only be undertaken with 80% commitment from all 
school staff, with all parties agreeing to sustain implementation efforts for at least three years. 

Administrator support. One of the strongest predictors of SWPBS sustainability is 
administrator support at both the school- and district- level (Coffey & Horner, 2012). A recent 
study assessing school personnel perceptions of critical features important to SWPBS 
sustainability found that perceived administrative support had the strongest impact on 
sustainability (McIntosh, Predy, et al., in press). Administrative support assures school staff that 
implementation will be supported by allocating resources (e.g., time, incentives, training), 
communicating expectations, and addressing competing practices that may decrease resources 
(Blase & Fixsen, 2004). A state or district leadership team that secures resources and political 
support, in addition to coordinating training, coaching, and evaluation, also highlights the 
importance of SWPBS at a district-level (Sugai & Homer, 2006). 

Implementer skill. The potential for effectiveness is dependent on whether the practice 
is both evidence-based and implemented with a high degree of fidelity (McIntosh, Mercer, et al., 
in press). SWPBS practices have been formulated over the last 50 years from a strong 
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foundation of empirically validated research (Homer, Sugai, & Anderson, 2010). However, if 
school personnel are only taught the procedures of the practice without understanding why it 
works and how it differs from their original beliefs and underlying assumptions about how 
students learn, teachers may either modify surface features (e.g., activities, materials) or merely 
add the new practice onto existing efforts without changing classroom norms (Cobum, 2003). 
High fidelity can then transform procedural understanding into conceptual understanding, as 
implementers learn how to integrate the practice into the local school culture while retaining the 
features critical to its effectiveness (Vaughn, Klingner, & Hughes, 2000). 

Teaming. Recent studies have noted effective team functioning as an important 
variable predicting SWPBS sustainability (Coffey & Horner, 2012; McIntosh, Mercer, et al., in 
press; McIntosh, Predy, et al., in press). A representative school-based leadership team of 6 to 10 
individuals responsible for leading SWPBS implementation is recommended (Sugai & Homer, 
2002). Team members attend annual training events, develop materials to support 
implementation, provide continued staff development, evaluate the fidelity of SWPBS 
implementation, and regularly meet to analyze school- wide behavior management systems and 
data (Lewis & Sugai, 1999). In addition to team expertise, McIntosh, Predy, and colleagues (in 
press) also reported that school personnel rated organized and regular meetings to be a critical 
feature of sustained implementation. 

Use of data. The use of data-based problem solving has been identified in the literature 
as an essential feature promoting SWPBS sustainability, as it provides a concrete and visible 
framework for systematically assessing the usefulness, effectiveness, and efficiency of a practice 
(Coffey & Homer, 2012; McIntosh, Mercer, et al., in press). Observing visible change in 
student behavior is often a powerful motivator in influencing teachers’ attitudes toward a 
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practice (Guskey, 1986). School personnel can also maintain the priority and relevance of the 
practice in the midst of a changing school environment as the practice is adapted to fit within the 
local culture of the classroom, is extended into novel areas (e.g., stakeholders, settings; Han & 
Weiss, 2005) or is made more efficient by allocating resources based on ongoing assessment of 
the costs of the process (McIntosh et al., 2009). SWPBS emphasizes the importance of 
systematic and ongoing documentation of the needs of school personnel, the fidelity of SWPBS 
implementation, and the impact of SWPBS on student behaviour. It is recommended that data 
be regularly reviewed at each team meeting, used to guide planning and decision-making, and 
frequently reported to all school personnel and other stakeholders (McIntosh et al., 2009). 

Ongoing technical assistance. Initial and ongoing technical assistance (i.e., training and 
coaching) has been identified as a critical factor in achieving high fidelity and sustainability 
(Coffey & Homer, 2012; McIntosh, Mercer, et al., in press). Access to this support allows the 
continued refinement of implementation, as school personnel can successfully contextualize the 
practice when responding to the demands of a changing host environment (Baker, Gersten, 
Dimino, & Griffiths, 2004). Initial training can generally involve a district coach involved in 
monthly meetings with the school team to build the knowledge of school personnel in SWPBS 
implementation. In addition, school teams from different schools and districts can create a 
professional community to support SWPBS implementation efforts. 

Efficient Measurement of Variables Important to Sustainability 

The previous studies yielded important information regarding variables that can be 
targeted to enhance sustainability, but these studies utilized external evaluations or additional 
measures to identify these variables. A more efficient approach would involve existing measures 
that school teams already use in SWPBS implementation, if such measures identify specific 
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critical features that are most strongly related to future fidelity of SWPBS implementation. This 
type of approach might also indicate more efficient strategies to promote sustainability, such as 
modifying initial or ongoing training to provide an additional focus on the SWPBS critical 
features that are most predictive of sustained implementation. 

Purpose of the Study 

The purpose of this study was to investigate the extent to which critical features from an 
existing measure of fidelity of SWPBS implementation predicted sustained fidelity of 
implementation and student outcomes. This study evaluated the predictive power of a self-report 
measure of fidelity of implementation in different SWPBS systems (school-wide, non-classroom, 
classroom, individual) on the levels of overall SWPBS implementation and problem behavior 3 
years later. Although technical assistance focusing on all aspects of SWPBS implementation 
may be effective, it would be especially useful to identify whether specific aspects of SWPBS 
are more predictive of sustainability and therefore could receive additional focus and resources 
to maximize sustained implementation. Two research questions were addressed: 

1. To what extent do school personnel ratings of implementation of SWPBS systems 
significantly predict sustained implementation and levels of problem behavior? 

2. Within any statistically significantly predictive SWPBS systems, which critical features 
of these systems significantly predict sustained implementation? 

Method 

Participants and Settings 

The respondents included school personnel from 261 schools across the United States 
who reported SWPBS fidelity data during a 3 year period. Although no definitive data regarding 
the first year of SWPBS implementation for each school were available, the median first year of 
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reporting any implementation or student outcomes data to the National Center on PBIS (a proxy 
for minimum years of SWPBS implementation) was 2004-05 (range = 2000-01 to 2006-07), 3 
years before the start of the study, and 27% of the sample were in their first year of 
implementation at the start of the study. Criteria for inclusion in this study specified that schools 
were administered the PBIS Self-Assessment Survey (SAS) in 2006-07 and the School-wide 
Benchmarks of Quality (BoQ) in 2009-10. In addition, 188 schools (72%) had ODR data that 
were used in a supplemental analysis. School demographic data from the National Center for 
Education Statistics were available for 98% of the sample. Of the demographic data available, 
most schools were located in Illinois (68%) and Oregon (23%), with the rest of the schools 
located in Colorado (3%), Idaho (2%), Michigan (2%), New York (1%), South Carolina (1%), 
and North Carolina (1%). Thirty-eight percent of the schools were Title 1 eligible, and an 
average of 47% of the students in each school was receiving free or reduced price meals. Most 
schools had fewer than 300 students (69%), and the sample contained 70% elementary schools, 
22% middle schools, 6% high schools, and 2% unspecified. The schools represented a range of 
communities (15% urban, 25% small/large city, 28% suburban, 4% semi-rural, 11% rural, and 
18% unspecified). The average ethnic makeup of each school was 54% European American, 
23% Hispanic, 17% African American, 5% Asian American, and 1% American Indian students. 
Measures 

PBIS Self-Assessment Survey (SAS). The SAS (Sugai, Horner, & Todd, 2003) is 
completed by school personnel to self-report fidelity of implementation of SWPBS in 4 settings 
in their school. It is a 43-item survey with 4 sections: School-wide Systems (across all settings 
in the school, 15 items); Non-classroom Systems (in settings such as cafeterias, playgrounds, and 
hallways; 9 items), Classroom Systems (in classrooms; 11 items), and Individual Systems 
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(support for individual students with chronic problem behaviors; 8 items). A critical SWPBS 
feature in a particular setting is measured by a single item on the SAS. Respondents self-report 
the implementation status of each critical feature (in place, partially in place, or not in place) to 
identify their perceptions regarding whether it is currently in place at their school. Individual 
responses are aggregated to provide school-level data in terms of percent of features in place at 
the individual item and systems (i.e., subscale) level. 

Psychometric data for the SAS are available from 2 studies. Hagan-Burke and 
colleagues (2005) reported strong internal consistency for the overall scale. Safran (2006) 
further reported strong internal consistency for the overall measure and moderate to strong 
internal consistency for the SWPBS setting subscales. Furthermore, the results provided a 
valuable means of facilitating staff participation, generating discussion, and linking results to 
action planning. In the current study, SAS subscale scores were used to measure prior 
implementation for each of the SWPBS settings (i.e., School-wide, Non-classroom, Classroom, 
Individual) in 2006-07. Prior implementation was indicated through the percent of critical 
features self-reported as in place at each school. Sample reliability for the SAS subscale scores 
was strong, as indicated by high alpha coefficients (a range = .86 to .97). 

Benchmarks of Quality (BoQ). The BoQ (Kincaid, Childs, & George, 2005) is a 53- 
item evaluation of fidelity of SWPBS implementation. The BoQ consists of 10 subscales: PBS 
Team (4 items), Faculty Commitment (3 items), Effective Discipline Procedures (7 items), Data 
Entry (5 items), Expectations and Rules (5 items), Reward System (8 items), Lesson Plans (6 
items), Implementation Plans (7 items), Crisis Plans (3 items), and Evaluation (5 items). A 
coach who is external to the school (e.g., district-level personnel) completes the BoQ using his or 
her knowledge of the school and the scoring guide, which provides operational definitions of the 
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ratings for each item. The maximum possible BoQ total score is 100. A BoQ total score of 70 or 
more indicates adequate fidelity of SWPBS implementation. 

Psychometric properties for the BoQ were examined by Cohen and colleagues (2007), 
who reported strong 1-week test-retest reliability (r = .94) and 2-week interrater reliability (r = 
.87). The total score correlation of the BoQ with the School-wide Evaluation Tool (SET; Horner 
et al., 2004), a widely used research-based SWPBS fidelity of implementation tool, was .51, 
indicating moderate concurrent validity. High predictive validity for the BoQ was established, as 
schools with higher BoQ total scores (> 70%) for 2 years of implementation had significantly 
decreased rates of office discipline referrals. The BoQ total score was used in this current study 
as a measure of sustained SWPBS fidelity of implementation. Sample reliability for the BoQ 
total score was strong, as indicated by a high coefficient alpha (a = .90). 

Office Discipline Referrals. In a supplemental analysis of a subset of the sample, levels 
of problem behavior at each school were indicated by Office Discipline Referrals (ODRs), 
standardized forms that are completed by school staff to document incidents of behavior. 

Previous research has shown ODRs to be valid indicators of overall levels of problem behavior 
in schools (Irvin, Tobin, Sprague, Sugai, & Vincent, 2004). In this study, ODRs were used to 
indicate sustained student outcomes of implementing SWPBS. 

Procedure 

Data extracted for this study included SAS scores in 2006-07, BoQ scores in 2009-10, 
and ODR data in 2009-10 from an extant database from Educational and Community Supports at 
the University of Oregon (University of Oregon, 2011). These schools had signed agreements 
with the Center on PBIS to allow their data to be available for research. 


Analyses 
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Three sets of regression analyses were conducted to investigate the extent to which self- 
reported prior implementation predicted sustained SWPBS implementation and student 
outcomes. In all analyses, subscale or item scores from the SAS administered in 2006-07 served 
as predictor variables, and either the BoQ total score or school-level number of ODRs in 2009-10 
served as the outcome variable. Before the main analyses, a preliminary analysis assessed 
whether the first year of reporting data to the Center on PBIS significantly predicted either 
outcome variable. It was not a statistically significant predictor, and as a result, it was not 
included in subsequent analyses. Next, a multiple regression analysis was conducted to evaluate 
the extent to which prior implementation of each SWPBS setting (i.e., School- wide, Non- 
classroom, Classroom, Individual) predicted sustained implementation, as measured by the BoQ 
total score in 2009-10, and sustained student outcomes, as measured by number of ODRs at a 
subset of schools in 2009-10. Finally, a multiple regression analysis was conducted to analyze 
specific critical features of any SWPBS setting identified in the first analysis that significantly 
predicted sustained implementation. Outliers were removed from the analysis if the scores 
exceeded the critical Mahalanobis distance (a statistical test for extreme outliers; Tabachnick & 
Fidell, 2007). Evaluations of the Trimmed Mean and Cook’s Distance did not reveal any further 
problems with outliers. All predictor variables were mean-centered to enhance interpretability of 
results and mitigate the effects of high predictor intercorrelations. 

Results 

Descriptive statistics for the BoQ total score and the SAS subscale scores for each of the 
SWPBS settings presented in Table 1 indicate that sustained SWPBS implementation in 2009-10 
was adequate for most schools ( M = 83%, SD = 1 1.40, range = 38 to 100%). There were strong, 


statistically significant intercorrelations between all of the SAS setting subscales. Each of the 
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SAS setting subscales had weak to moderate, statistically significant positive correlations with 
subsequent sustained implementation. 

Results of the multiple regression analyses displayed in Table 2 indicate that the overall 
model involving self-reported fidelity of prior implementation in all SWPBS settings was 
statistically significant, F( 4, 254) = 8 . 18,79 < .001 and explained 10% of the variance in 
sustained implementation. When evaluating the individual predictors, only prior implementation 
in Classroom Systems was a statistically significant predictor, f> = .28, /> < .05. Similarly, the 
only statistically significant predictor of level of ODRs was Classroom Systems, /? = -.43, p < .05 

For the second research question, the critical features of the Classroom Systems subscale 
(the only statistically significant predictor) were examined. Descriptive statistics for the SAS 
prior implementation (Classroom Systems) critical features displayed in Table 3 indicate that 
defining ( M = 83%) and teaching (M = 76%) expected behaviors were rated as strongly 
implemented overall. The other critical items had weak to moderate implementation (M = 47 - 
67%). There were statistically significant small to medium positive correlations between each 
Classroom Systems feature and the overall BoQ score. There were strong, statistically 
significant intercorrelations between all Classroom System features. 

Table 4 indicates that multiple regression results for the overall model predicting 
sustained implementation from the critical features of the Classroom Systems subscale was 
statistically significant and explained 14% of the variance, F(ll, 234) = 4.61, p < .001. 
Significant predictors at the .05 level included: expected behaviors are acknowledged regularly 
if) = .29); instruction and curriculum materials are matched to student ability {[:> = .22); and 
teachers have opportunities for access to assistance and recommendations (fi = -.21). The 
negative beta weight for access to assistance, combined with its positive correlation to the 
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outcome may be an artifact of regression or may potentially indicate the effects of net 
suppression (Cohen & Cohen, 1975). 

Discussion 

The current study explored the extent to which implementer perceptions of their 
implementation of critical SWPBS features predicted sustained implementation 3 years later. 
The sample was composed of 261 schools implementing SWPBS practices for a range of years. 
Self-reported fidelity of implementation was assessed using the PBIS Self-Assessment Survey 
(SAS). The School-wide Benchmarks of Quality (BoQ) was administered 3 years later as an 
indicator of sustained fidelity of SWPBS implementation, and ODRs were assessed in a 
subsample of schools as an indicator of sustained student outcomes. Results of multiple 
regression analyses indicated that the SAS was a statistically significant predictor of sustained 
implementation, and prior implementation in Classroom Systems was a small but statistically 
significant unique predictor of both sustained SWPBS implementation and levels of problem 
behavior. Within the individual features of the Classroom Systems subscale, acknowledging 
expected student behaviors regularly and positively, instructionally matching curriculum to 
student need, and access to additional support were also small but statistically significant unique 
predictors of sustained implementation. 

Practices Predicting Sustained SWPBS Implementation 

School-wide versus classroom practices. The finding that the Classroom System 
subscale was a stronger predictor of school- wide implementation and student outcomes than the 
School-wide and Non-classroom Systems subscales may be somewhat surprising to general 
audiences but is supported by the theory that the actions of individual teachers are most 
important to sustainability (Han & Weiss, 2005). Students spend the vast majority of their 
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school day in the classroom. As core SWPBS implementers, classroom teachers have regular 
and ongoing opportunities to implement SWPBS practices in their classrooms by creating 
environments that increase the likelihood of students learning both academic and behavioral 
skills. Although SWPBS is a school-wide approach, both the quality and durability of 
implementation may be contingent on the extent to which individual teachers implement SWPBS 
classroom practices with high fidelity. 

However, the strong correlations between each of the SAS subscales show that 
implementation in each area is strongly linked to implementation in the others. Thus, an 
overwhelming focus on classroom implementation to the detriment of school- wide 
implementation is not supported by these findings. Creating a common school- wide framework 
of expectations, values, and systems of support is critical in developing the school into an 
effective host environment by building internal capacity within the school environment and 
school personnel to implement effective interventions in all SWPBS settings, including 
classrooms. As a result, although developing a common underlying framework of SWPBS 
values and expectations is critical, it may also be effective to focus on helping school personnel 
translate these core values into their everyday classroom teaching practices, which may improve 
both fidelity of implementation and student outcomes, both of which enhance sustainability. 

Regular positive reinforcement. The results of the current study highlight the critical 
importance of regular acknowledgement of expected student behaviors as a key instructional 
variable that predicts sustained SWPBS implementation. Although defining and teaching 
expected behaviors were not significant unique predictors of sustained implementation, these two 
features had the highest implementation scores in the Classroom Systems subscale. Positive 
acknowledgement of appropriate behavior, a central tenet of SWPBS practices (Sugai & Horner, 
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2006), is the next step in increasing the likelihood of the desired behavior in the future while also 
fostering positive student-teacher interactions. However, despite consistent recommendations 
for increasing ratios of positive to negative statements in the classroom (e.g., four positive 
acknowledgements for each error correction; Sugai, 2002), research regularly shows higher 
ratios of negative to positive statements for social behavior (Jenson, Olympia, Farley, & Clark, 
2004). Within a SWPBS approach, many schools implementing SWPBS use tangible rewards 
(e.g., tickets that can be exchanged for school supplies or social activities) as a systems-level 
intervention for increasing the implementer-Ievel behavior of specific verbal praise. Thus, using 
a school-wide acknowledgement system with a continuum of varied, age-appropriate strategies 
provides a consistent and effective mechanism to increase rates of positive reinforcement (Sugai, 
Homer, & McIntosh, 2008). Reviews of research have indicated that the use of rewards is only 
associated with a decrease in intrinsic motivation when they are expected, provided only once, 
and not directly tied to level of performance (Akin-Little, Eckert, Lovett, & Little, 2004; 
Cameron, Banko, & Pierce, 2001). Witzel and Mercer (2003) further noted that explicit 
acknowledgement of the student’s behavior may be more important than the reward itself. 

Hence, a system of tangible rewards may be most effective when it promotes regular and specific 
teacher acknowledgement of student behaviors. Consistent with research (Andreou & McIntosh, 
2013), the focus on positive reinforcement and visible improvement seen by teachers when using 
acknowledgement systems effectively seems to be a critical mechanism in sustaining SWPBS. 

Matching instruction and materials to student ability. Matching academic 
instruction and curriculum materials to the needs of the students was also a significant unique 
predictor of sustained implementation. For example, teachers using differentiated instruction are 
able to adapt their curriculum and materials to the varying levels of student skill in their 
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classroom with the goal of maximizing the potential of each learner in a given subject area 
(Tomlinson, 2005). In addition, matching instructional demands to student skill levels has been 
shown to reduce problem behavior, even in the absence of additional behavior support (Filter & 
Homer, 2009; Lee, Sugai, & Horner, 1999; Preciado, Horner, & Baker, 2009). An appropriate 
match between instructional materials and student skills is pivotal for sustainability because it 
can prevent student frustration and subsequent problem behavior while maximizing student 
success. Thus, supporting teachers’ academic instructional skills can be effective in preventing 
problem behavior, ensuring academic success, and creating a positive context in which SWPBS 
can be sustained. When a range of valued student outcomes (i.e., academic and behavior 
performance) and teacher outcomes (i.e., facing fewer instances of problem behavior) are 
maximized, the priority for sustaining the practice is enhanced (Han & Weiss, 2005). 

Access to assistance and recommendations. Findings from the current study show 
that access to additional support was significantly positively correlated with sustained 
implementation, but had a negative relationship when controlling for the previous two 
instructional practices. The positive bivariate correlation between access to assistance and 
sustained implementation obtained in this study is consistent with findings that receiving 
additional support, in the form of coaching or peer networking, is associated with improved 
teaching (Rose & Church, 1998). However, when positive reinforcement and instructional 
matching were held constant in the final model, access to additional support was associated with 
lower levels of implementation. One potential explanation of the negative relation may be that it 
is a spurious artifact of the regression. However, the results may also be interpretable if there is a 
plausible theoretical explanation for the relation (Messick & Van de Geer, 1981). One potential 
hypothesis is that access to additional support is predictive of sustained SWPBS implementation 
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when it focuses on improving key instructional practices, but simply providing access to 
additional support that does not improve these key instructional practices may not enhance 
sustainability. For example, receiving ineffective coaching (either focused on less relevant 
teaching variables or coaching that did not result in improved instruction) may have been a 
barrier to subsequent sustained efforts. In addition, some schools may also have been receiving 
additional, mandated instructional support (e.g., as a persistently underperforming school) that 
indicated potential external challenges in classroom management or school culture. However, 
future research must be conducted to evaluate this hypothesis. 

Limitations 

One important limitation was the limited variability in BoQ total scores, as most schools 
had reached adequate fidelity of implementation by 2009-10. Schools that did not report fidelity 
of implementation data during the 3-year period were not included in the sample, as their 
outcomes data were not available. As such, these results may not be applicable to predicting 
whether schools abandon SWPBS entirely. In addition, different variables affect fidelity of 
implementation at different time points (McIntosh, Predy, et al., in press). Self-reported fidelity 
scores may have been influenced by social desirability or lack of knowledge of quality 
implementation. Finally, because the second research question used single items to assess 
implementation of critical features, results of the classroom systems analysis should be 
interpreted with caution. 

Future Research 

Current findings indicate that the level of SWPBS implementation in classrooms 
significantly predicts sustained SWPBS efforts and student outcomes, but it is also clear that 
other variables facilitate sustained implementation. More studies that use additional (non- 
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extant), more comprehensive measures of contextual variables related to sustainability are 
needed to capture the full range of variables predicting sustained implementation. Research may 
also investigate factors predicting SWPBS abandonment. Future studies could also track fidelity 
of implementation across multiple years to identify different facilitators or barriers to sustained 
implementation at different phases of implementation. 

Implications for Practice 

Development of school-wide and classroom practices. A relatively common 
approach to SWPBS implementation includes implementing systems to define, teach, and 
acknowledge school-wide expectations in school-wide and non-classroom settings and then 
leaving it to classroom teachers to extend these common school- wide practices to their 
classrooms (McKevitt & Braaksma, 2008). The logic behind this approach is that classroom 
teachers may be more likely to implement SWPBS in the classroom if they see positive effects in 
shared areas, where it is easier to build consensus. The danger is that classroom teachers may 
never implement practices in their classrooms without a collective endeavor and specific 
classroom systems consultation. Or worse, they may believe that SWPBS practices are intended 
only for non-classroom areas and not the classroom. Training that focuses on improving the 
understanding of key principles of the practice, in addition to information on the intervention’s 
effectiveness, may improve teacher acceptability of the practice by increasing expectations, 
intentions, and motivation to implement the practice (Han & Weiss, 2005). Yet building teacher 
acceptance of SWPBS may not be enough, as school personnel may also need to be shown how 
to translate the core SWPBS components into their daily routines. These findings provide initial 
support for building consensus to implement SWPBS in classroom settings as soon as it is 
implemented in non-classroom settings. Access to additional support to address SWPBS 
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implementation in the classroom may also promote full classroom implementation when 
associated with improved teaching practices. However, this recommendation requires research 
validation. 

Ongoing data collection and visibility of results. SWPBS emphasizes the importance 
of systematic documentation of the needs of school personnel, fidelity of implementation, and its 
effects on valued outcomes (Sugai & Horner, 2006). Regular data collection and interpretation 
of student performance facilitates accurate identification of student needs and the development of 
quality instructional practices to maximize both academic and behavioral student success. 
Additionally, generating charts and figures linking changes in student behavior directly to 
fidelity of implementation would help build consensus to implement SWPBS in classroom 
settings. Comparing the results of self-reported fidelity of implementation and independent 
evaluations of SWPBS can also provide important information regarding both the level of 
implementation and school personnel understanding of core SWPBS principles. Completion of a 
yearly needs assessment, such as the SAS, allows for implementer-level ownership in action 
planning for sustained implementation. In addition, although fidelity of implementation in 
school-wide settings is commonly assessed, some PBIS fidelity of implementation measures 
have recently been revised to include more classroom- specific items (Kincaid, Childs, & George, 
2010; Sugai, Homer, Lewis-Palmer, & Rossetto Dickey, 2011). These measures may be more 
effective in measuring sustainability in both classroom and non-classroom areas. 
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Table 1. Descriptive Statistics for BoQ Total Score and SAS Subscale Scores 


Measure 

Mean 

SD 

Range 

BoQ (2009-10) 

83% 

11.40 

38 - 100% 

SAS (2006-07) 

School-wide 

61% 

19.30 

7 - 100% 

Non-Classroom 

56% 

20.28 

8 - 100% 

Classroom 

62% 

16.18 

15 - 100% 

Individual 

42% 

18.27 

3 - 100% 
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Table 2. Prediction of Sustained SWPBS Implementation and Office Discipline Referrals 
(ODRs) in 2009-10 from Prior Implementation (All SWPBS Systems) in 2006-07 



b 

SE b 

P 

R~ 

Adjusted R~ 

Implementation Model (n = 261) 




H*** 

IQ*** 

Constant 

83 

.68 




School-wide 

-.16 

.11 

-.26 



Non-classroom 

.14 

.10 

.25 



Classroom 

.19 

.09 

.28* 



Individual 

.05 

.07 

.08 



ODR Model (n = 188) 




.04 

.02 

Constant 

819.51 

70.83 




School-wide 

11.13 

11.42 

.23 



Non-classroom 

.68 

10.17 

.02 



Classroom 

-24.63 

10.21 

-.43* 



Individual 

3.54 

7.42 

.07 




Note. *p < . 05. **/?<. 01. ***/?<. 001. 
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Table 3. Descriptive Statistics for SAS Prior Implementation (Classroom Systems) Critical 
Feature Scores in 2006-07 

SAS Prior Implementation (Classroom Systems; 2006-07) Mean SD 

1. Expected student behavior and routines in classrooms are 83% 15.38 

stated positively and defined clearly 

2 . Problem behaviors are defined clearly 67 % 19.19 

3. Expected student behavior and routines in classrooms are 76% 19.71 

taught directly 

4. Expected student behaviors acknowledged regularly 59% 22.77 

(positively reinforced; >4 positives to 1 negative) 

5. Problem behaviors receive consistent consequences 50% 21.23 

6. Procedures for expected and problem behaviors are 57% 23.70 

consistent with school- wide procedures 

7. Classroom-based options exist to allow classroom 52% 21.24 

instruction to continue when problem behavior occurs 

8. Instruction and curriculum materials are matched to student 61% 19.28 

ability (math, reading, language) 

9. Students experience high rate of academic success (>75%) 47% 22.54 

10. Teachers have regular opportunities for access to assistance 48% 20.05 

and recommendations 

11. Transitions between instructional and non-instructional 55% 20.79 


activities are efficient and orderly 



Running head: FEATURES PREDICTING SUSTAINED SWPBS IMPLEMENTATION 30 


Table 4. Prediction of Sustained SWPBS Implementation in 2009-10 from Prior Implementation 
(Classroom Systems) Critical Features in 2006-07 

b SE b /? /? Adjusted R 2 


Model .18*** 14 *** 


Constant 

83 

.70 


Expected Behaviors Defined 

-.05 

.10 

-.07 

Problem Behaviors Defined 

.10 

.09 

.16 

Expected Behaviors Taught 

-.11 

.09 

-.16 

Expected Behaviors Acknowledged 

.16 

.07 

.29* 

Consequences 

-.09 

.07 

-.16 

Procedures 

-.01 

.08 

-.02 

Options Exist 

.11 

.07 

.18 

Instruction/Materials Match Ability 

.14 

.06 

.22* 

Academic Success 

-.03 

.05 

-.06 

Access to Assistance 

-.13 

.06 

-.21* 

Transitions Efficient 

.12 

.07 

.21 


Note. *p < .05. **p < .01. ***p < .001. 



